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INSPECTOR  GENERAL 
DEPARTMENT  OF  DEFENSE 
400  ARMY  NAVY  DRIVE 
ARLINGTON,  VIRGINIA  22202-4704 


November  24,  2010 

MEMORANDUM  FOR  DIRECTOR,  OPERATIONAL  TEST  AND  EVALUATION 

AUDITOR  GENERAL,  DEPARTMENT  OF  THE  ARMY 

SUBJECT:  Live  Fire  Testing  of  Light  Tactical  Wheeled  Vehicles  was  Effective  for  the  Portions 
Completed  (Report  No.  D-20 1 1  -0 1 9) 

We  are  providing  this  report  for  your  information  and  use,  We  determined  that  the  Army’s  live 
fire  testing  of  the  up-armored  High  Mobility  Multi-Purpose  Wheeled  Vehicle  was  effective  for 
the  portions  completed.  We  also  found  that  DOD  oversight  of  the  live  fire  test  and  evaluation 
was  effective  for  the  portions  of  the  oversight  process  completed.  We  are  making  no 
recommendations  and  do  not  require  a  written  response.  Therefore,  we  are  publishing  this  report 
in  final  form. 

We  appreciate  courtesies  extended  to  the  staff.  Please  direct  questions  to  me  at  (703)  604-9201 
(DSN  664-9201). 


Richard  B.  Jolliffe 
Assistant  Inspector  General 
Acquisition  and  Contract  Management 
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November  24,  2010 


Results  in  Brief:  Live  Fire  Testing  of  Light 
Tactical  Wheeled  Vehicles  was  Effective 
for  the  Portions  Completed 


What  We  Did 

We  determined  whether  the  Army 
effectively  planned,  executed,  and  evaluated 
High  Mobility  Multi-Purpose  Wheeled 
Vehicle  (HMMWV)  live  fire  testing  and 
whether  DOD  exercised  adequate  live  fire 
test  and  evaluation  oversight  of  the  Army’s 
HMMWV  Program.  This  report  is  the 
second  in  a  series  of  reports  on  the  Army’s 
efforts  to  develop,  test,  and  acquire  armor 
solutions  for  light  tactical  wheeled  vehicles. 

What  We  Found 

The  U.S.  Army  Test  and  Evaluation 
Command’s  (ATEC)  live  fire  testing  of  the 
up-armored  HMMWV  was  effective  for  the 
portions  completed.  Specifically,  ATEC 
planned  a  live  fire  test  strategy  for  the 
HMMWV  Program  that  identified  required 
documents  needed  to  determine  system  and 
crew  survivability.  ATEC  provided  the 
required  live  fire  planning  documents  to  the 
Office  of  the  Director,  Operational  Test  and 
Evaluation,  for  approval  and  ATEC 
executed  tests  in  accordance  with  the 
approved  live  fire  test  plans.  ATEC  has  not 
yet  completed  the  up-armored  HMMWV 
live  fire  test  and  evaluation  report  but  will 
provide  it,  upon  completion,  for  the 
Director,  Operational  Test  and  Evaluation’s 
evaluation  of  the  up-armored  HMMWV  live 
fire  test  and  evaluation. 

The  Office  of  the  Director,  Operational  Test 
and  Evaluation’s  live  fire  test  and  evaluation 
oversight  of  the  Army’s  up-armored 
HMMWV  was  effective  for  the  portions  of 
the  oversight  process  completed. 


Specifically,  the  Director,  Operational  Test 
and  Evaluation,  placed  the  up-annored 
HMMWV  on  the  Office  of  the  Secretary  of 
Defense  Test  and  Evaluation  Oversight  List 
in  2006  for  live  fire  oversight  and  has 
reviewed,  as  required,  the  Army’s 
up-armored  HMMWV  live  fire  test  planning 
documents.  The  Director,  Live  Fire  Test 
and  Evaluation,  stated  that  the  Director, 
Operational  Test  and  Evaluation,  will 
publish  a  HMMWV  live  fire  test  and 
evaluation  report  by  the  end  of  2010. 

What  We  Recommend 

The  report  contains  no  recommendations. 

Management  Comments 

We  do  not  require  a  written  response  to  this 
report. 


Ml  151-Model  HMMWV  With 
Fragmentation  Kit  #7 


Source:  http://www.army.mil 
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Introduction 

Audit  Objectives 

Our  objectives  were  to  determine  whether  the  Anny  effectively  planned,  executed,  and  evaluated 
High  Mobility  Multi-Purpose  Wheeled  Vehicle  (HMMWV)  live  fire  testing  and  whether  DOD 
exercised  adequate  live  fire  test  and  evaluation  (LFT&E)  oversight  of  the  Army’s  HMMWV 
Program.  See  the  appendix  for  a  discussion  on  scope  and  methodology  and  prior  coverage 
related  to  the  audit  objectives. 

Background 

This  is  the  second  in  a  series  of  reports  on  the  Army’s  efforts  to  develop,  test,  and  acquire  annor 
solutions  for  light  tactical  wheeled  vehicles.  The  first  report  in  the  series,  DOD  Inspector 
General  (IG)  Report  No.  D-2010-039,  “Recapitalization  and  Acquisition  of  Light  Tactical 
Wheeled  Vehicles,”  January  29,  2010,  addressed  the  Army’s  efforts  to  develop  and  acquire  the 
Risk  Reduction  Vehicle,  or  XM1 166,  and  the  Next  Generation  Expanded  Capacity  Vehicle,  or 
ECV2.  We  initiated  these  audits  as  a  result  of  information  gathered  while  conducting  the  audit 
that  led  to  DOD  IG  Report  No.  D-2009-030,  “Marine  Corps  Implementation  of  the  Urgent 
Universal  Needs  Process  for  Mine  Resistant  Ambush  Protected  Vehicles,”  December  8,  2008. 
Specifically,  we  were  presented  with  infonnation  that  led  us  to  question  the  survivability  testing 
of  the  HMMWV  against  mines  and  improvised  explosive  devices  (IEDs). 

Live  Fire  Test  and  Evaluation 

LFT&E  is  a  test  process  to  evaluate  the  vulnerability  and  lethality  of  a  conventional  weapon  or  a 
conventional  weapon  system.  LFT&E  provides  insights  into  a  weapon  system’s  ability  to 
continue  to  operate  or  fight  after  being  hit  by  enemy  threat  systems.  Live  fire  testing  provides  a 
way  to  examine  the  damage  inflicted  on  materiel  and  personnel  and  an  opportunity  to  assess  the 
effects  of  complex  environments  that  crews  are  likely  to  encounter  in  combat.  Conducting  live 
fire  tests  is  very  complex,  and  many  factors  have  to  be  considered  in  the  LFT&E  of  tactical 
wheeled  vehicles.  Those  factors  include,  but  are  not  limited  to:  the  type  of  munition  used, 
standoff  distance  of  the  vehicle  to  the  explosive,  projectile  orientation  angle,  shotlines,  the  size 
of  the  fragment-simulating  projectile  (FSP),1  where  to  fire  FSPs  at  the  vehicle,  soil  conditions, 
whether  to  conduct  a  test  that  would  cause  catastrophic  or  significant  damage  to  the  vehicle,  and 
differences  between  the  individual  vehicles. 

Test  Methodology  Concerns 

During  the  audits  of  the  “Marine  Corps  Implementation  of  the  Urgent  Universal  Needs  Process 
for  Mine  Resistant  Ambush  Protected  Vehicles”  and  “Recapitalization  and  Acquisition  of  Light 
Tactical  Wheeled  Vehicles,”  we  interviewed  vehicle  survivability  subject  matter  experts  who 
expressed  concerns  about  the  U.S.  Army  Test  and  Evaluation  Command’s  (ATEC’s)  live  fire 
testing  methods. 


1  An  FSP  is  a  specific  fragment  simulator  type  based  on  a  standardized  cylindrical  projectile  with  a  chiseled  nose.  It 
is  designed  to  be  fired  from  a  gun  to  simplify  armor  testing. 
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Specifically,  the  subject  matter  experts  were  concerned  that  ATEC  personnel: 

•  used  FSPs  to  test  annored  vehicles  that  did  not  accurately  represent  fragments  from  IEDs 
detonated  close  to  a  vehicle, 

•  incorrectly  used  composition-4  (C-4)  as  an  explosive  equivalent  to  trinitrotoluene  (TNT), 
and 

•  performed  IED  tests  to  evaluate  vehicle  annor  protection  in  soil  that  did  not  represent  the 
soil  found  in  theater  operations. 

Results 

To  address  the  vehicle  survivability  subject  matter  experts’  concerns  about  ATEC’s  live  fire 
testing  methods,  we  reviewed  ATEC’s  LFT&E  planning  and  execution,  use  of  FSPs,  use  of  C-4 
in  lieu  of  TNT,  and  whether  the  soil  used  in  live  fire  testing  was  representative  of  the  soil  found 
in  theater  operations.  We  also  evaluated  Director,  Operational  Test  and  Evaluation’s 
(DOT&E’s)  LFT&E  oversight  of  the  HMMWV  Program. 

Planning  and  Execution  of  Up-Armored  HMMWV  Live  Fire  Test  and 
Evaluation  was  Effective  for  the  Portions  Completed 

ATEC’s  live  fire  testing  of  the  up-armored  HMMWV  was  effective  for  the  portions  completed. 
Specifically,  ATEC  planned  a  live  fire  test  strategy  for  the  HMMWV  Program  that  identified 
required  documents  needed  to  detennine  system  and  crew  survivability.  ATEC  provided  the 
required  live  planning  documents  to  the  Office  of  the  DOT&E  for  approval  and  ATEC  executed 
tests  in  accordance  with  the  approved  live  fire  test  plans.  ATEC  has  not  yet  completed  the  up- 
armored  HMMWV  LFT&E  report  but  will  provide  it,  upon  completion,  for  DOT&E’s  evaluation 
of  the  up-armored  HMMWV  LFT&E. 

Frag  men  t-Simula  ting  Projectiles 

Data  collected  by  the  U.S  Army  Aberdeen  Test  Center  and  the  U.S.  Army  Research  Laboratory 
(ARL)  revealed  that  the  tactic  of  employing  artillery  shells  as  surface-laid  IEDs,  as  opposed  to  an 
airburst  detonation,  resulted  in  an  increase  in  the  number  of  the  bigger  fragments  threatening  a 
vehicle.  An  ARL  representative  stated  that  the  20mm  FSP  represents  the  97th  percentile  of 
fragments,  by  mass,  from  a  surface-laid  152mm  artillery  shell  detonation.  Based  on  ARL’s  test 
data,  we  confirmed  that  the  20mm  FSP  represents  the  97th  percentile  of  fragments,  by  mass,  from 
a  surface-laid  152mm  artillery  shell  detonation.  The  Anny  used  a  combination  of  FSP  tests, 
full-up  system  level  vulnerability  tests,2  and  modeling  and  simulation3  to  detennine  the 
up-armored  HMMWV’ s  capabilities  and  limitations  against  surface-laid  artillery  shell-based  IED 
threats  and  other  threats. 

Explosive  Equivalency 

An  ATEC  representative  stated  that,  from  May  2004  to  November  2006,  ATEC  used  the  C-4 
explosive  conversion  as  a  substitute  for  a  TNT  mine  explosive  when  performing  underbody  tests 

2 

“  Full-up  system  level  vulnerability  tests  are  tests  conducted  on  a  complete  system  loaded  or  equipped  with  all  the 
dangerous  materials  that  normally  would  be  on  board  in  combat  (including  flammables  and  explosives)  and  with  all 
critical  subsystems  operating  that  could  make  a  difference  in  determining  the  test  outcome.  These  tests  use 
munitions  likely  to  be  encountered  in  combat. 

3  We  did  not  evaluate  the  models  or  simulations  the  Army  uses  in  the  live  fire  test  and  evaluation  process. 
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of  the  HMMWV  variants.  ATEC  used  C-4  to  support  the  HMMWV  requirement  to  provide 
underbody  protection  against  mine  blasts  to  the  HMMWV  variants.  The  C-4  conversion  used  by 
ATEC  created  smaller  sized  craters  during  the  test  events  than  the  crater  sizes  created  by  TNT 
mines  in  theater.  ATEC  representatives  stated  that  they  used  the  C-4  explosive  substitute 
because  it  was  the  best  they  could  do  at  the  time.  Although  ATEC  incorrectly  used  C-4  as  an 
explosive  equivalent  to  TNT  when  performing  underbody  mine  testing,  the  TNT  mine  weight 
requirement  in  the  September  2004  HMMWV  Operational  Requirements  Document  was  less 
than  two-thirds  of  the  mine  explosive  weight  experienced  in  theater  operations. 

By  2008,  ATEC  changed  the  testing  methodology  from  using  the  C-4  explosive  equivalent  as  a 
substitute  for  the  TNT  underbody  mine  threat  to  using  a  cast  TNT  mine.  ARL  developed  cast 
TNT  mines  as  threat  surrogates  for  the  actual  mine.  As  documented  in  its  live  fire  test  planning 
documents,  ATEC  plans  to  use  an  approved  threat  surrogate  during  live  fire  testing  instead  of  the 
C-4  explosive  conversion  in  its  underbody  mine  blast  testing. 

Soil  Testing 

An  ARL  representative  stated  that  he  discussed  with  officials  from  Developmental  Test 
Command4  the  possibility  of  conducting  a  study  to  detennine  whether  test  procedures  in  a  new 
soil  would  be  repeatable.  He  stated  that  ARL  can  complete  this  type  of  study.  Further,  the 
representative  stated  that  current  tests  conducted  in  consistent  soil  retain  a  10  to  25  percent 
variability  in  blast  effect  from  shot  to  shot.  ARL  is  also  in  the  process  of  developing  a 
comparison  of  the  different  types  of  soil  that  range  from  dense,  saturated  clay  to  loose,  sifted 
sand.  From  this  comparison  study,  ARL  is  working  on  a  rough  conversion  factor  that  would 
allow  testers  to  take  real  observed  test  results  done  in  one  soil  condition  and  use  that  data  to 
estimate  probable  results  of  the  same  test  as  if  it  had  been  done  in  another  soil  type.  Results  of 
tests  would  still  need  to  be  compared  to  older  vehicle  tests  done  in  the  current  standard  test  soil. 

A  representative  from  the  U.S.  Army  Corps  of  Engineers  Engineer  Research  Development 
Center  provided  a  2009  report  which  examined  the  detonation  of  explosives  in  soil.  He  also 
stated  that  the  Center  is  in  the  process  of  conducting  a  soil  study  that  will  focus  on  the  tactical 
implications  of  differences  in  soil  conditions.  The  study  will  identify  how  soil  conditions  affect 
ground  radar  used  in  IED  detection  systems  and  how  the  depth  of  an  IED  affected  the  blast  to 
vehicle  underbellies.  The  U.S.  Army  Corps  of  Engineers  Engineer  Research  Development 
Center  surveyed  10  to  11  sites,  and  the  United  Kingdom  team  members  are  gathering  data  from 
5  additional  sites. 

Effective  Oversight  by  the  Director ;  Operational  Test  and  Evaluation 
Oversight 

The  Office  of  the  DOT&E’s  LFT&E  oversight  of  the  Army’s  up-armored  HMMWV  was 
effective  for  the  portions  of  the  oversight  process  completed.  The  up-armored  HMMWV  was 
added  to  the  Office  of  the  Secretary  of  Defense  Test  and  Evaluation  Oversight  List  in  2006,  and 
the  vehicle  remained  under  DOT&E  oversight  for  live  fire  testing  in  2010.  DOT&E  reviewed 
and  approved  up-armored  HMMWV  test  planning  documents,  as  required.  As  a  part  of  its 


4  Developmental  Test  Command  is  a  part  of  ATEC  and  is  responsible  for  planning,  conducting,  and  reporting 
developmental  tests. 
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assessment  of  the  survivability  of  the  up-armored  HMMWV,  DOT&E  originally  planned  to  use 
ATEC’s  HMMWV  Live  Fire  Report.  However,  on  April  23,  2010,  DOT&E  issued  a 
memorandum  to  the  Service  test  community  stating  that  DOT&E  will  publish  an  independent 
report  before  the  completion  of  the  test  agency  report,  if  necessary  to  ensure  the  timely  reporting 
of  operational  and  live  fire  test  results.  Therefore,  DOT&E  will  start  its  assessment  of 
the  survivability  of  the  up-armored  HMMWV  whether  or  not  ATEC  has  provided  its  HMMWV 
LFT&E  report.  The  Director,  Live  Fire  Test  and  Evaluation,  stated  that  DOT&E  will  publish  a 
HMMWV  LFT&E  report  by  the  end  of  2010. 
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Appendix.  Scope  and  Methodology 

We  conducted  this  performance  audit  from  October  2009  through  October  2010  in  accordance 
with  generally  accepted  government  auditing  standards.  Those  standards  require  that  we  plan 
and  perfonn  the  audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit 
objectives. 

The  objectives  of  the  audit  were  to  detennine  whether  the  Army  effectively  planned,  executed, 
and  evaluated  HMMWV  live  lire  testing  and  whether  the  DOD  exercised  adequate  live  lire 
testing  oversight  of  the  Army’s  HMMWV  Program. 

In  our  review,  we  included  the  HMMWV  variants  that  were  part  of  the  Office  of  the  Secretary  of 
Defense  Test  and  Evaluation  Oversight  List  from  2006  to  2010.  The  HMMWV  variants  on  the 
Oversight  List  consisted  of  the  Ml  15 1  family  of  vehicles. 

We  did  not  include  the  Next  Generation  Expanded  Capacity  Vehicle,  or  ECV2,  in  our  review  of 
DOD’s  live  lire  testing  oversight  of  the  HMMWV  Program.  The  Army  considered  the  Next 
Generation  Expanded  Capacity  Vehicle  as  a  part  of  the  HMMWV  Program  and  merely  an 
upgrade  to  the  existing  HMMWV.  Therefore,  DOT&E  listed  the  Next  Generation  Expanded 
Capacity  Vehicle  as  “HMMWV  ECV2”  on  its  Office  of  the  Secretary  of  Defense  Test  and 
Evaluation  Oversight  List.  However,  as  discussed  in  DOD  IG  Report  No.  D-20 10-039, 
“Recapitalization  and  Acquisition  of  Light  Tactical  Wheeled  Vehicles,”  we  do  not  agree  that  the 
Next  Generation  Expanded  Capacity  Vehicle  is  a  HMMWV  because  it  had  less  than  30  percent 
of  parts  in  common  with  the  Expanded  Capacity  Vehicle -model  HMMWV  and  would  be  built  on 
a  separate  production  line. 

To  accomplish  the  objectives  of  the  audit,  we  reviewed  the  Office  of  the  Secretary  of  Defense 
Test  and  Evaluation  Oversight  Lists  from  2006  through  2010;  live  fire  test  planning  documents, 
such  as  the  HMMWV  Live  Fire  Test  and  Evaluation  Strategy,  Detailed  Test  Plan,  Event  Design 
Plan,  and  Pre-Shot  Prediction  Report;  the  HMMWV  Operational  Requirements  Document;  and 
various  DOD  studies  and  reports  from  January  1973  through  March  2010. 

We  contacted  staff  in  the  Offices  of  the  Director,  Operational  Test  and  Evaluation;  Joint 
Improvised  Explosive  Device  Defeat  Organization;  U.S.  Army  Test  and  Evaluation  Command; 
and  U.S.  Anny  Research  Laboratory. 

Use  of  Computer-Processed  Data 

We  did  not  use  computer-processed  data  to  perform  this  audit. 


The  HMMWV  M1151  family  of  vehicles  includes  the  M1151,  M1152,  and  the  M1165. 
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Use  of  Technical  Assistance 

The  Technical  Assessment  Directorate,  Office  of  the  Deputy  Inspector  General  for  Policy  and 
Oversight,  assisted  the  audit  team  by  analyzing  data  related  to  the  U.S.  Army  Test  and 
Evaluation  Command  testing  methodologies  used  during  the  live  fire  test  and  evaluation  of  the 
HMMWV  Program. 

Prior  Coverage 

During  the  last  5  years,  the  Government  Accountability  Office  (GAO),  the  DOD  IG,  and  the 
Army  Audit  Agency  have  issued  eight  reports  discussing  the  HMMWV ;  add-on  armor  for 
tactical  wheeled  vehicles;  production,  delivery,  and  installation  of  truck  armor;  tactical  wheeled 
vehicle  strategies;  and  recapitalization  and  acquisition  of  light  tactical  wheeled  vehicles. 

Unrestricted  GAO  reports  can  be  accessed  over  the  Internet  at  http://www.gao.gov.  Unrestricted 
DOD  IG  reports  can  be  accessed  at  http://www.dodig.mil/audit/reports. 

Unrestricted  Army  reports  can  be  accessed  from  .mil  and  gao.gov  domains  over  the  Internet  at 
https://www.aaa.army.mil/. 

GAO 

GAO  Report  No.  GAO-06-274,  “Defense  Logistics:  Lack  of  a  Synchronized  Approach  between 
the  Marine  Corps  and  Army  Affected  the  Timely  Production  and  Installation  of  Marine  Corps 
Truck  Armor,”  June  22,  2006 

GAO  Report  No.  GAO-06- 160,  “Defense  Logistics:  Several  Lactors  Limited  the  Production  and 
Installation  of  Army  Truck  Annor  during  Current  Wartime  Operations,”  March  22,  2006 

DODIG 

DOD  IG  Report  No.  D-20 10-039,  “Recapitalization  and  Acquisition  of  Light  Tactical  Wheeled 
Vehicles,”  January  29,  2010 

DOD  IG  Report  No.  D-2009-046,  “Procurement  and  Delivery  of  Joint  Service  Armor  Protected 
Vehicles,”  January  29,  2009 

DOD  IG  Report  No.  D-2009-030,  “Marine  Corps  Implementation  of  the  Urgent  Universal  Needs 
Process  for  Mine  Resistant  Ambush  Protected  Vehicles,”  December  8,  2008 

DOD  IG  Report  No.  D-2008-089,  “Planning  Armor  Requirements  for  the  Lamily  of  Medium 
Tactical  Vehicles,”  May  9,  2008 

DOD  IG  Report  No.  D-2007-107,  “Procurement  Policy  for  Armored  Vehicles,”  June  27,  2007 

Army 

Army  Audit  Agency  Report  No.  A-2007-0234-ALM,  “Tactical  Wheeled  Vehicle  Strategy, 
Deputy  Chief  of  Staff  G-8  and  Program  Executive  Office  Combat  Support  and  Combat  Service 
Support,”  September  26,  2007 
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